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INTRODUCTION 


Neuropsychological  assessments  are  useful  for  detecting  both  impaired  and  optimal 
cognitive  functioning.  Neuropsychological  tests  have  not  changed  greatly  since  the  days 
when  Halstead  and  Reitan  introduced  standardized  cognitive  testing.1  Now,  as  then,  the 
major  drawbacks  in  cognitive  assessment  are  the  time  it  takes  to  administer  the  battery 
as  well  as  the  expertise  required  to  administer,  score,  and  interpret  the  tests.  Several 
computerized  neurocognitive  assessment  tools  (NCAT)  based  upon  standardized 
neuropsychological  tests  of  speed  and  accuracy  have  been  developed  to  overcome 
these  limitations.  The  Automated  Neuropsychological  Assessment  Metrics  (ANAM)  was 
the  first  widely  utilized  automated  battery.2  Initially  sponsored  by  US  Department  of 
Defense  (DoD)  and  later  adopted  for  baseline  testing  of  service  members  prior  to 
deployment,  ANAM  is  widely  used  with  over  1 .5  million  tests  administered  in  the  past 
five  years.3 

In  January  2009,  the  U.S.  Navy  Bureau  of  Medicine  and  Surgery  identified  a  need  to 
enhance  existing  battlefield  concussion  assessment  and  requested  the  development  of 
a  durable,  portable,  and  field-hardened  NCAT  to  provide  a  practical  means  to  conduct 
neurocognitive  and  psychological  assessment  in  field  deployment  settings.  The  purpose 
of  combining  neurocognitive  and  psychological  assessment  was  to  permit  more 
comprehensive  evaluation  of  the  broad  range  of  problems  that  may  be  encountered 
during  combat  deployment.  The  Defense  Automated  Neurobehavioral  Assessment 
(DANA)  was  designed  to  meet  this  need.  The  DANA  was  specifically  developed  to  be 
field  deployable  under  a  wide  range  of  environmental  conditions. 

A  recent  effort  reported  that  the  specific  subtests  with  the  DANA  battery,  simple  reaction 
time  (SRT1),  simple  reaction  time  repeated  (SRT2),  procedural  reaction  time  (PRO), 
coded  substitution  (CDS),  and  coded  substitution  delayed  (CDD),  on  average  had 
moderate  to  good  (i.e.,  0.50-0.75)  concurrent  validity  with  the  ANAM  for  throughput  and 
median  reaction  time.4  The  test-retest  reliability  of  the  simple  reaction  time  for  DANA 
was  superior  to  the  ANAM,  0.75  vs. 0.64  and  0.70  vs. 0.36,  for  median  reaction  time  and 
throughput  respectively.4 

A  second  study  reported  the  internal  consistency  and  reliability  of  DANA’s  hardware  and 
software  functioning  under  challenging  field  conditions.  These  environments  included 
the  arctic  (Thule  Air  Force  Base — Greenland  in  the  winter),  jungle  (U.S.  Marine  Corps 
Jungle  Warfare  Training  Center — Okinawa,  Japan,  in  the  summer),  at  altitude  (U.S. 
Marine  Corps  Mountain  Warfare  Training  Center — Bridgeport,  CA,  approximately  3,000 
m),  desert  (U.S.  Marine  Corps  Desert  Warfare  Training  Center — Twentynine  Palms,  CA, 
in  the  summer),  and  shipboard  (USS  George  Washington  during  high  seas  in  the 
Western  Pacific).  The  split-half  reliability  correlations  for  DANA  subtests  were  within 
acceptable  ranges,  and  were  comparable  to  those  reported  for  the  ANAM.5  However, 
the  study  did  not  use  the  same  participants  across  field  conditions,  and  the  reported 
reliability  estimates  cannot  be  used  to  assume  reliability  across  the  field  conditions,  but 
should  be  interpreted  as  an  estimate  of  test-retest  reliability. 
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Thus,  the  objective  of  this  effort  was  two-fold: 

1 .  To  estimate  the  internal  consistency  of  the  DANA  within  a  baseline  ambient 
environmental  condition. 

2.  To  estimate  the  reliability,  both  absolute  and  relative,  of  the  DANA  across  four 
simulated  environmental  conditions  in  the  same  personnel. 


METHODS 

Participants 

The  NEDU  Institutional  Review  Board  (IRB)  approved  protocol  number  14- 
23/40063, ’’Evaluation  of  the  Defense  Automated  Neurobehavioral  Assessment  (DANA) 
in  Extreme  Environments.”  The  participants  for  this  effort  consisted  of  sixteen  male  U.S. 
Navy  personnel  from  NEDU.  Navy  personnel  were  recruited  for  this  effort  via  direct 
solicitation  by  the  Task  Leader  through  email  correspondence  or  in  a  group  setting  with 
the  NEDU  IRB  ombudsman  present.  Participants  were  briefed,  provided  with  the 
protocol,  encouraged  to  read  it,  and  provided  with  the  opportunity  to  ask  questions  prior 
to  giving  written  informed  consent.  Participants  were  assigned  identification  numbers 
after  they  enrolled.  Files  and  log  sheets  were  identified  by  participant  number,  study 
session,  and  date.  A  list  linking  participant  number  to  participant  name  was  stored 
securely  by  the  investigators  during  the  study  and  was  destroyed  once  data  collection 
and  analysis  was  complete. 

Experimental  Design  and  procedures 

A  repeated  measures  (2  x  4)  experimental  approach  was  employed.  Two 
administrations  (i.e.,  tests)  of  the  NCAT  were  performed  per  testing  session.  Each  of  the 
four  testing  sessions  consisted  of  one  of  the  following  simulated  environmental 
conditions: 

1 .  baseline/ambient  (BSL;  72  ±  3°F,  50%  humidity), 

2.  cold  (CLD;  37  ±  3°F,  target  of  50%  relative  humidity), 

3.  hot/dry  (HD;  113  ±  3°F,  target  of  20%  relative  humidity), 

4.  hot/humid  (HH;  104  ±  3°F,  target  of  80%  relative  humidity), 

The  simulated  environmental  conditions  were  selected  to  induce  thermal  discomfort,  but 
not  result  in  any  change  in  core  temperature.6'9  Wet-bulb  globe  temperatures  (WBGT) 
indexes  were  calculated  for  each  condition  and  used  to  determine  the  safe  exposure 
limit  for  each  condition  via  Table  B2-A-2  from  OPNAVINST  5100. 19E  (see  Table  1 
below)  for  Physical  Exertion  Level  (PHEL)  Curve  I  (i.e.,  light  work).10 

The  participants  performed  at  least  five  practice  sessions  of  the  DANA  on  a  handheld 
computing  device  prior  to  testing  (range  5-32  days)  to  familiarize  with  the  assessment 
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and  to  minimize  the  risk  of  any  learning  and/or  practice  effects.  A  breakdown  of  the 
practice  and  testing  sessions  is  found  in  Table  1. 


Table  1.  Test  conditions  for  neurocognitive  testing. 


Session 

Environmental  condition 

NCAT 

No.  of 

administrations 

Practice 

- 

DANA  Brief 

At  least  5 

Testing 

Baseline  /  ambient 

DANA  Brief 

2 

Cold 

DANA  Brief 

2 

Hot  /  dry 

DANA  Brief 

2 

Hot  /  humid 

DANA  Brief 

2 

Note:  NCAT  =  neurocognitive  assessment  tool;  No.  =  number. 


Four  participants  were  each  randomly  assigned  to  one  of  four  groups.  The  four 
members  of  the  group  were  tested  together  during  each  environmental  condition.  Each 
group  had  at  least  a  day  between  each  testing  session.  All  participants  were  required  to 
wear  minimal  clothing  (t-shirt,  shorts,  socks,  and  shoes).  Light  gloves  and  ear  coverings 
were  allowed  for  the  cold  environmental  condition.  Participants  were  required  to  refrain 
from  alcohol  for  24  hours  and  exercise  for  12  hours  prior  to  each  testing  session.  Upon 
arrival  at  the  laboratory,  participants  were  encouraged  to  void  their  bladders.  They  were 
then  weighed  and  instrumented  for  measurement  of  core  temperature  (Tc),  skin 
temperature  (TSk),  and  heart  rate  (HR). 

A  brief  refresher  was  then  given  on  how  to  complete  the  DANA  test  battery  where 
participants  were  instructed  to  respond  as  rapidly  and  accurately  as  possible. 
Participants  were  instructed  to  complete  two  consecutive  administrations  with  each 
administration  lasting  approximately  10-15  minutes.  Participants  were  instructed  to 
immediately  start  the  second  administration  upon  completion  of  the  first  and  to  sit 
quietly  until  all  participants  had  completed. 

After  the  procedure  was  fully  explained,  the  participants  entered  the  environmental 
chamber  which  was  pre-set  to  one  of  the  four  simulated  environmental  conditions. 
Towels  were  supplied  for  comfort  purposes.  Once  the  environmental  chamber  doors 
where  closed,  participants  sat  patiently  watching  movies  for  25  minutes  prior  to  start  of 
the  first  assessment.  Thus,  the  total  testing  time  was  approximately  45-55  minutes  for 
each  environmental  condition. 

Equipment  and  instrumentation 

The  DANA  is  an  NCAT  that  consists  of  a  user  interface,  multiple  automated  cognitive 
tests,  data  management  and  reporting  capabilities,  and  executive  menu.  The  DANA 
battery  was  given  on  a  Trimble  NOMAD®  handheld  computing  device  using  a  stylus  for 
consistency.  The  NOMAD  runs  a  custom  version  of  the  Android  Operating  System  and 
has  a  color  3.5  inch,  pressure  sensitive  touch  screen.  It  has  been  suggested  that  this 


3 


format  is  optimally  suited  for  tests  of  speed  and  accuracy  as  there  is  less  variability  in 
the  test  procedures  than  those  encountered  in  standard  desktop-based  assessments.11 

The  DANA  Brief  Battery  was  used  in  this  study  to  assess  neurocognitive  performance. 
The  Brief  Battery  consists  of  six  subtests:  Simple  Reaction  Time  (SRT),  Code 
Substitution  (CDS),  Procedural  Reaction  Time  (PRO),  Spatial  Processing  (SPD),  Go- 
No-Go  (GNG),  and  Code  Substitution  Recall  (CDD).  Descriptions  of  these  subtests  can 
be  found  in  Table  2.  All  tests  are  self-explanatory  and  begin  with  written  instructions.  To 
minimize  learning  and  practice  effects,  test  stimuli  are  generated  at  random  and  each 
test  has  built  in  practice  trials  embedded  in  every  assessment  prior  to  the  actual  test 
administration. 

Table  2.  DANA  Brief  Battery  Subtest  Descriptions 


Subtest 

Test  Description 

SRT 

This  test  measures  pure  reaction  time  and  cognitive  processing  time.  The  participant  taps  on  the  location 
of  a  bulls-eye  as  quickly  as  possible  each  time  it  appears. 

CDS 

This  test  assesses  immediate  memory  and  attention.  A  sequence  of  single  symbol-digit  pairs  are  shown 
below  a  key,  &  the  participant  indicates  it  matches  the  one  in  the  key. 

PRO 

This  test  targets  decision-making  capabilities.  One  of  the  four  numbers  is  displayed  at  a  time  for  three  sec 
prior  to  the  display  of  the  next  number.  Participants  are  asked  to  differentiate  whether  the  number  that 
appears  on  the  screen  is  “2  or  3”  or  “4  or  5”. 

SPD 

This  test  assesses  spatial  manipulation.  Pairs  of  four-bar  histograms  are  presented  simultaneously  on  the 
screen,  one  rotated  90  degrees  from  the  other.  The  participant  determines  if  the  two  histograms  are  the 
same  or  different. 

GNG 

The  test  assesses  speed  and  accuracy  of  targets,  omissions,  and  commissions  in  order  to  derive  a 
sensitivity  metric,  as  found  in  continuous  performance  tasks.  A  house  is  presented  on  the  screen  with 
several  windows.  Either  a  “friend”  (green)  or  “foe”  (white)  appears  in  the  window.  The  respondent  must 
push  a  button  only  when  a  “foe”  appears. 

CDD 

This  test  assesses  short-term  memory.  After  a  delay,  symbol-digit  pairs  are  presented  without  the  key. 

The  participant  indicates  whether  or  not  the  pairing  was  included  in  the  key  that  was  presented  in  the 
earlier  Code  Substitution  learning  section. 

Physiological  and  environmental  data  were  continuously  monitored  during  each  testing 
session.  The  following  instrumentation  was  used  during  each  testing  session: 

•  Trimble  NOMAD  with  DANA  Brief  loaded; 

•  A  T-8  data  logger  for  recording  Tc  and  TSk  data; 

•  A  Tc  sensor  inserted  15  cm  beyond  the  anal  sphincter, 

•  TSk  sensors  placed  on  the  head,  arm,  pinky  finger,  foot,  calf,  thigh,  and  abdomen; 

•  Polar  HR  monitor  strapped  around  the  participant’s  thorax; 

•  Extech  HT  30  Heat  Stress  WBGT  Meter. 

Mean  TSk  was  calculated  from  a  7-point  measuring  system  using  the  following  average 
weighting  coefficients  for  each  region.12  The  formula  used  to  calculate  mean  was: 

Mean  Tsk  =  0.07  (head)  +  0.14  (arm)  +  0.07  (foot)  +  0.13  (calf)  +  (0.19)thigh  +  0.35  (abdomen) 
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All  data,  minus  the  WBGT  Meter,  were  monitored  and  logged  in  LabVIEW  in  real  time. 
These  data  were  not  used  for  data  analysis  purposes,  but  rather  to  monitor  participant 
safety  and  to  ensure  that  the  environmental  chamber  was  functioning  properly. 

Data  analysis 

Data  processing.  The  technique  we  used  to  clean  the  dataset  was  a  valid,  yet  extremely 
conservative,  approach  involving  lower  and  upper  thresholds  or  cutoffs  to  remove 
contaminant  reaction  times  outside  of  the  normal  range  of  responses  (i.e.,  outliers). 
Outliers  can  distort  estimates  of  mean  reaction  time  and  standard  deviation  in  reaction 
time  as  well  as  reduce  the  power  of  analyses  of  variance.13  The  lower  cut-off  was  used 
to  remove  ‘fast’  outliers  that  are  considered  anticipatory  and  too  fast  to  be  precise 
and/or  deliberate  responses.14  The  upper  cutoff  was  used  to  remove  ‘slow’  outliers  that 
were  a  result  of  lapses  in  concentration  and  was  set  at  a  point  at  which  few  responses 
would  fall.14  Both  the  upper  and  lower  cut-offs  were  set  using  normative  data  for  military 
members  for  the  ANAM.15  The  cut-offs  have  been  used  in  previous  studies  and  are 
presented  in  Table  3. 11 


Table  3.  Data  Cleaning  Cutoffs  for  Reaction  Times. 


Subtest 

“Fast” 

“Slow” 

SRT 

<  150  ms 

900  ms 

PRO 

2000  ms 

GNG 

1500  ms 

SPD 

<  300  ms 

5000  ms 

CDS 

3000  ms 

CDD 

4000  ms 

Note:  ms  =  milliseconds. 


The  remaining  data  were  selected  for  further  analysis  and  percent  correct  and 
throughput  were  calculated  for  each  subtests: 


%  correct  = 


#  correct  trials 
#  trials 


Mean  Throughput  = 


%  correct 

Mean  correct  Reaction  Time 


(1) 

(2) 


Participants  with  missing  data  as  a  result  from  termination  of  testing  were  removed  from 
further  analysis  to  ensure  that  partial  data  was  not  evaluated. 

The  normality  of  average  throughput  scores  were  statistically  verified  with  both  Lilliefors 
and  Jarque-Bera  tests  (p  >  0.05).  Descriptive  statistics  were  computed  for  mean 
throughput  for  each  DANA  subtest  and  each  environmental  condition. 

Internal  consistency.  Data  from  both  administrations  in  the  baseline  condition  were 
combined  to  act  as  two  parts  or  halves  of  a  single  administration  for  the  internal 
consistency  estimations.  Cronbach’s  alpha  (a)  coefficient  was  used  to  measure  the 
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internal  consistency  (i.e.,  homogeneity)  of  the  throughput  scores  for  consecutive 
administrations  for  each  of  the  DANA  subtests  within  the  baseline  condition.  The  metric 
is  calculated  as  follows: 

rk 

K"  «l  +  (k— l)r]) 


Whereas  k  is  the  number  of  items  considered  and  r  is  the  mean  of  the  inter-item 
correlation.  The  size  of  a  is  determined  by  the  number  of  items  in  the  scale  and  mean 
inter-item  correlation.  An  a  greater  than  or  equal  to  0.80  was  considered  “excellent”, 
values  between  0.79  and  0.70  were  considered  “adequate”,  and  those  below  0.69  were 
to  be  considered  poor.16 

Subtest  reliability.-  Data  from  the  first  assessment  in  each  of  the  environmental 
conditions  was  used  in  the  reliability  estimations.  The  relative  reliability  across  all  four 
environmental  conditions  was  estimated  using  a  two-way  random,  average  measures 
intra-class  correlational  coefficients  (ICC)  defined  by: 

BMS  -  MSE 

jrr  —  _ 

(2’k)  BMS  ,  «WMS  -  MSE ) 

n 

whereas  BMS  is  the  between-subjects  mean  square,  MSE  is  the  residual  mean  square 
error,  WMS  is  the  within-subjects  mean  square,  k  is  the  number  of  data  collection 
sessions  (i.e.,  environmental  conditions),  n  is  the  number  of  subjects,  all  obtained  from 
a  (1  x  4)  analysis  of  variance  (ANOVA)  with  repeated  measures  with  the  environmental 
condition  (4  levels)  as  the  within-subjects  variable.  Univariate  tests  and  simple  effects 
tests  were  conducted  to  follow  up  significant  main  effects  and  interactions. 

The  ICC  provides  an  estimate  of  relative  reliability  for  the  consistency  of  measurement. 
It  is  a  ratio  of  the  variance  due  to  differences  between  the  participants  (the  signal)  and 
the  total  variability  in  the  data  (i.e.,  noise).  Essentially,  it  is  a  unit-less  measure  of  the 
test  error  in  regard  to  between  subject  variability.  Thus,  it  is  reflective  of  the  ability  of  a 
test  to  differentiate  between  different  individuals.17  An  ICC  greater  than  or  equal  to  0.90 
was  considered  “very  high”,  values  between  0.80  and  0.89  were  considered  “high”, 
values  between  0.70  and  0.79  were  considered  “adequate”,  values  between  0.60  and 
0.69  were  considered  “marginal”,  and  those  below  0.59  were  to  be  considered  poor.16  A 
value  >  0.90  is  recommended  for  clinical  decision  making  and  a  value  >  0.70  is 
conventionally  seen  as  acceptable  for  research  purposes.18 

The  standard  error  of  the  measurement  (SEM),  or  ‘typical  error’,  was  used  to  quantify 
absolute  reliability.17,19  It  carries  the  same  units  of  measurement  as  the  variable  of 
interest  and  the  interpretation  of  the  SEM  centers  on  the  assessment  of  reliability  within 
subjects.14  The  SEM  is  the  square  root  of  the  MSE  found  in  the  ICC  ANOVA  calculation 
(SEM  =  a/M5I).17,19 
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Subtest  reliability  between  each  environmental  condition  and  the  baseline.  Relative 
reliability  was  estimated  using  three  two-way  random,  single  measure  ICC  estimated 
from  the  results  of  three  separate  (1  x  2)  analysis  of  variance  (ANOVA)  models  with 
repeated  measures  with  environmental  condition  (2  levels)  as  the  with  in-subjects 
variable.  Univariate  tests  and  simple  effects  tests  were  conducted  to  follow  up 
significant  main  effects  and  interactions.  Absolute  reliability  or  within-participant 
variation  was  estimated  using  the  SEM. 


RESULTS 

A  total  of  16  male  military  personnel  participated  in  this  study.  The  mean  age  in  years 
was  34  (standard  deviation  [SD]  =  7.5).  The  mean  height  in  centimeters  was  178.1  (SD 
=  8.3).  The  mean  body  mass  in  kilograms  was  88.9  (SD  =  10.8).  The  mean  time  from 
the  start  of  the  first  practice  session  to  the  completion  of  the  fourth  administration  was 
23.7  days  (SD  =  5.6).  The  mean  time  to  complete  all  four  assessments  was  15.53  days 
(SD  =  4.8)  with  an  average  of  3.9  (SD  =  1 .2)  days  between  each  administration.  Not  all 
participants  completed  all  four  environmental  conditions  due  to  early  termination  of 
testing,  equipment  malfunction,  or  participant  no  shows.  Thus,  there  were  a  total  of  13 
participants  who  had  complete  datasets.  Descriptive  statistics  across  environmental 
conditions  are  presented  in  Table  4. 


Table  4.  Descriptive  statistics  for  DANA  subtest  throughput  scores 


Subtest 

N 

BSL 

CLD 

HD 

HH 

CDS 

13 

53.49  (7.60) 

54.42  (6.09) 

57.18  (6.25) 

55.62(10.29) 

CDD 

13 

54.13(10.47) 

52.03  (6.02) 

59.59  (6.39) 

60.82  (9.56) 

GNG 

13 

58.26  (1.69) 

56.97  (3.04) 

57.61  (2.37) 

57.79  (1.83) 

PRO 

13 

104.34  (3.04) 

100.78  (9.54) 

105.48  (9.11) 

105.57(11.09) 

SPD 

13 

40.84  (7.80) 

40.93(11.86) 

41.40  (8.27) 

41.56  (9.45) 

SRT 

13 

236.69  (30.90) 

232.34  (21.35) 

229.97  (31.50) 

221.67  (40.58) 

SRT2 

13 

239.36  (27.70) 

225.16  (25.44) 

224.19(35.40) 

220.98  (38.05) 

Note:  All  values  are  throughput  scores  represented  as  mean  (standard  deviation). 


Internal  consistency.  As  shown  in  Table  5,  the  mean  internal  consistency  (a)  of  the 
DANA  subtests  within  the  baseline  condition  was  found  to  be  acceptable  with  mean 
values  (i.e.,  across  environmental  conditions)  ranging  from  poor  for  the  ODD  subtest 
(.21 )  to  excellent  for  other  five  subtests  (CDS,  GNG,  PRO,  SPD,  SRT,  and  SRT2). 
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Table  5.  DANA  internal  consistency  and  reliability  metrics  across  environmental  conditions. 


Subtest 

or 

ICC 

ICC  95%  Cl 

F 

SEM 

CDS 

,95a 

.91  b’c 

[.46  -.89] 

1.98 

4.07 

CDD* 

.21 

.74° 

[.14  -.72] 

5.73 

6.37 

GNG 

.82a 

.79° 

[.20  -.76] 

1.79 

1.44 

PRO 

,94a 

,90b' c 

[.45  -.88] 

2.25 

5.41 

SPD 

.88a 

,92b’ c 

[.52  -.90] 

0.07 

4.86 

SRT 

,87a 

,88c 

[.40  -.86] 

1.49 

18.66 

SRT2 

.89a 

.86c 

[.34  -.83] 

2.14 

20.13 

Note:  *  =  p  value  <  .007  for  significance  after  adjusting  for  multiple  comparisons; 
a  =  meets  acceptable  or  higher  levels  of  internal  consistency  (>  .70); 
b  =  meets  threshold  for  clinical  decision  making; 
c  =  meets  acceptable  or  higher  levels  of  reliability  (>  .70). 

Subtest  reliability.-  As  shown  in  Table  5,  the  ICC  values  ranged  from  .74  to  .92  (95%  Cl 
lower  limits  >0.14  and  upper  limits  <  0.90).  All  of  the  subtests  reached  the 
conventionally  accepted  threshold  of  >  0.70,  with  CDS  (0.91),  PRO  (0.90),  and  SPD 
(0.92)  surpassing  the  very  high  mark.  The  subtest  with  the  lowest  ICC  value,  CDD 
(0.74),  had  significant  difference  between  the  environmental  condition  means  (F  =  5.75, 
p  <  .007).  Pairwise  comparisons  revealed  that  mean  for  the  cold  environmental 
condition  (M=52.03,  SE=1 .67)  was  significantly  lower  than  the  means  for  the  HD 
(M=59.59,  SE=1.77)  and  HH  (M=60.82,  SE=2.65)  environmental  conditions.  The  SEM 
values  ranged  from  a  low  of  1 .44  for  PRO  to  high  of  20.13  for  SRT2. 


DISCUSSION 

The  purpose  of  this  effort  was  to  empirically  examine  specific  psychometric  properties  of 
the  DANA,  an  NCAT  that  was  developed  as  a  durable,  portable,  and  field-hardened 
alternative  to  the  ANAM.  The  internal  consistency  and  the  reliability,  both  relative  and 
absolute,  of  the  DANA  were  estimated  from  data  collected  from  16  active  duty  service 
members  currently  billeted  at  the  Navy  Experimental  Diving  Unit.  The  DANA  Brief  test 
battery  was  administered  eight  times  in  four  different  simulated  field  relevant 
environmental  conditions  over  a  period  of  24  days.  The  primary  findings  were  that  the 
subtests  within  the  DANA  Brief  test  battery  demonstrated  adequate  (0.79)  internal 
consistency  and  acceptable-to-excellent  (0.74  to  0.92)  relative  reliability  across  the  four 
simulated  environmental  conditions. 

Internal  consistency  (i.e.,  alpha  coefficients)  estimates  reliability  across  test  items  in 
terms  of  item  variance  and  covariance  during  a  single  administration  of  a  test  battery. 
The  mean  (i.e.,  across  environmental  conditions)  internal  consistency  value  for  each 
subtest  ranged  from  unacceptable-to-excellent  with  a  mean  of  0.79  with  only  the  CDD 
subtest  (0.21 )  falling  outside  of  the  excellent  range.  The  low  alpha  coefficient  of  the 
CDD  was  not  unexpected  as  the  participants  completed  two  consecutive 
administrations  of  the  DANA  Brief  test  battery  which  may  have  resulted  in  proactive 
interference.20,21  Proactive  interference  can  occur  following  repeated  administrations  of 
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similar  stimuli  (e.g.,  digit-symbol  pairings)  as  the  stimuli  learned  in  the  first 
administration  interferes  with  the  recall  of  the  second  stimuli.22  Essentially,  the  individual 
gets  confused  about  whether  the  digit-symbol  pairing  was  presented  during  the  second 
administration  or  the  first  administration. 

The  alpha  coefficients  presented  here  are  very  similar  to  previously  reported  values  for 
the  DANA  subtests  which  ranged  from  adequate-to-excellent  (mean  =  0.78)5  and  for 
values  reported  for  the  ANAM  which  ranged  from  poor-to-excellent  (mean  =  0.63).21  A 
high  degree  of  internal  consistency  is  imperative  for  any  NCAT  as  it  provides  confidence 
to  a  clinician  and/or  researcher  in  their  ability  to  interpret  the  results  as  a  reflection  of 
the  test  items.20  Guidelines  provided  by  Nunnally,  considers:  alpha  coefficients  >  0.70  to 
be  adequate  for  a  test  in  the  early  stages  of  development;  alpha  coefficients  between 
0.90  and  0.80  to  be  adequate  for  more  developed  tests  such  as  the  subtests  found  in 
the  DANA  and  ANAM  subtests;23  and  alpha  coefficients  >  0.90  to  be  indicative  of  a 
redundancy  among  items  within  a  test.24  However,  it  is  more  likely  the  alpha  coefficients 
>  0.90  in  the  current  study  were  not  a  result  of  the  redundancy  of  a  single  administration 
of  the  subtest,  but  simply  the  result  of  administering  two  test  batteries  consecutively,  as 
all  subtests  were  presented  twice.25  Additionally,  an  alpha  coefficient  >  0.80  can 
suggest  good  relative  reliability  because,  as  generalization  of  the  split-half  reliability, 
Cronbach’s  alpha  estimates  the  likelihood  of  obtaining  the  same  scores  under  multiple 
administrations.26 

Relative  reliability  has  been  argued  to  be  the  most  important  psychometric  property  of 
any  NCAT.27  The  DANA  subtests  ICCs  ranged  from  acceptable-to-very  high  with  a 
mean  of  0.86.  More  specifically,  all  of  the  subtests  reached  the  conventionally  accepted 
threshold  for  reliability  of  >  0.70.  However,  it  has  been  suggested  that  this  threshold  is 
not  strict  enough  for  the  purpose  of  individual  decision  making,  and  that  an  ICC  of  0.90 
or  higher  is  necessary  to  provide  useful  information  for  clinical  decision  making.27  Only 
the  CDS,  PRO,  and  SPD  subtests  surpassed  this  threshold.  The  ICCs  presented  here 
were  similar  to  previous  reports  for  DANA  subtests,  which  ranged  from  poor-to-very 
high  with  a  mean  of  0.85, 5  but  slightly  higher  than  those  reported  for  ANAM  subtest, 
which  ranged  from  poor-to-high  with  a  mean  of  0.74. 18  It  should  be  noted  that  in  the 
previous  reports,  three  of  the  DANA  subtests  reached  the  clinical  decision  making 
threshold,  but  none  of  the  ANAM  subtest  did.5,18  One  explanation  why  no  ANAM  subtest 
reached  the  0.90  mark  was  the  differences  in  the  test-retest  interval  used  in  the  studies. 
Dretsch  et  al18  used  an  interval  of  eight  days  which  was  twice  the  length  of  the  interval 
used  in  the  current  study  and  eight  times  as  long  as  the  interval  used  by  Lathan  et  al.5 
The  length  of  the  test-retest  interval  is  important  as  it  has  been  reported  that  shorter 
intervals  result  in  more  stable  tests  results  and  higher  relative  reliability.29  However,  it 
should  be  noted  that  if  the  intervals  are  too  short  (e.g.,  within  the  same  day)  it  can  result 
in  decreased  reliability  due  to  possible  practice  effects  29 

A  technique  that  has  been  used  to  classify  cognitive  testing  measures  using  both  the 
internal  consistency  and  relative  reliability  values  was  used  to  examine  NCAT 
performance.  Each  subtest  was  sorted  into  one  of  two  groups,  either  “most  reliable” 
group  or  the  “least  reliable”  group.  A  classification  as  “most  reliable”  indicates  that 
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clinicians  should  have  the  greatest  confidence  in  the  precision  of  scores  from  the  tests. 
In  order  to  be  classified  as  “most  reliable,”  the  subtest  had  to  have  an  alpha  coefficient 
between  0.85  and  0.99  and  an  ICC  between  0.75  and  0.99.30  In  order  to  be  classified  as 
“least  reliable,”  the  subtest  had  to  have  an  internal  consistency  value  less  than  0.80  and 
a  relative  reliability  below  0.60. 30  Using  these  criteria  the  current  study  and  Lathan  et  al 
classified  four  DANA  subtests  (CDS,  GNG,  PRO,  and  SRT2)  as  being  “most  reliable” 
and  no  subtest  as  “least  reliable.”5  The  only  discrepancy  between  the  two  studies  was 
that  the  current  study  classified  the  SPD  and  SRT2  subtest  as  “most  reliable”  whereas 
Lathan  et  al  did  not.  Using  data  from  Cernich  et  al,  three  ANAM  subtests  (STM  and 
CPT)  were  classified  as  “least  reliable”  and  no  subtest  as  “most  reliable.”21 

Although  this  study  attempted  to  correct  for  many  of  the  limitations  with  the  previous 
report  on  the  psychometric  properties  of  the  DANA,  there  were  still  limitations  pertaining 
to  the  current  study.  The  sample  size  was  homogenous  which  limits  the  applicability  to 
the  general  military  or  civilian  population.  A  second  limitation  was  the  relatively  short 
interval  of  ~  4  days  was  used  between  test  administrations  as  research  has  suggested 
that  shorter  test  intervals  result  in  higher  reliability  estimates.  Thus,  a  longer  interval  of 
two  weeks  or  a  month  may  provide  a  more  accurate  estimate  of  reliability.  A  third 
limitation  could  be  the  presence  of  learning  or  practice  effects.  Practice  effects  may  be 
represented  by  a  change  in  test  scores  from  one  administration  to  the  next  even  when 
no  intervention  is  involved.  However,  there  was  no  way  to  assess  for  the  presence  of 
any  practice  effects  without  a  test-retest  assessment  performed  in  each  environmental 
condition.  Again  it  should  be  noted  that  in  an  attempt  to  control  for  practice  effects 
stimuli  in  the  DANA  subtests  are  generated  at  random  and  each  subtest  has  built  in 
practice  trials  within  each  administration. 


CONCLUSIONS 

The  DANA  Brief  test  battery  was  administered  to  active  duty  personnel  at  the  NEDU  in 
four  different  simulated  field  relevant  environmental  conditions.  The  internal  consistency 
and  reliability  was  evaluated  and  it  was  found  that  DANA  subtests  demonstrated 
adequate  internal  consistency  and  acceptable-to-excellent  relative  reliability  across  the 
four  simulated  environmental  conditions.  Our  findings  add  to  the  current  body  of 
research  regarding  the  psychometric  properties  of  the  DANA;  however,  future  research 
should  focus  on  determining  the  validity,  both  construct  and  concurrent,  of  the  DANA 
subtests  against  results  of  similar  traditional  clinical  paper-and-pencil 
neuropsychological  tests. 
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APPENDIX  A 


Standard  Operating  Procedure  for  the  Defense  Automated  Neurobehavioral 
Assessment  (DANA)  vl.5.3  Application  on  the  Samsung  GALAXY  Note®  vlO.1 

The  purpose  of  this  Standard  Operating  Procedure  (SOP)  is  to  provide  clear  and 
detailed  instructions  on  how  to  operate  and  utilize  the  DANA  vl  .5.3  app  that  has  been 
pre-installed  onto  a  Samsung  GALAXY  Note®  vlO.1 .  These  procedures  are  used  to  set 
up  and  administer  the  DANA  cognitive  tests  to  potential  subjects.  These  procedures  are 
also  used  to  transfer  screening  data  to  a  Data  Management  PC  for  analysis. 

The  minimum  requirements  for  the  Data  management  PC  are  as  follows: 

•  Operating  system:  Windows  XP  (Service  Pack  2),  Windows  7,  or  Mac  OS  X, 

•  RAM:  3GB, 

•  Storage:  200MB  free, 

•  Processor:  X86  (intel  or  AMD),  2GHz, 

•  Ports:  1  USB  1.0, 

•  Display:  1024x768. 

The  following  SOP  requires  that  the  following  applications/packages/files  be  installed  on 
the  Data  Management  PC  using  the  DANA  v.1 .5.3  Quick  Reference  Guide  Section  B. 

•  Microsoft  Visual  C++  package  (Windows  XP  only). 

o  Launch  vcredistx86  (20110).exe  or  vcredistx64  (20110).exe  for  32-bit 
Windows  XP  and  64-bit  Windows  XP  respectively. 

•  The  Samsung  Android  USB  Driver  for  Windows 

o  Visit  the  following  website  to  download  the  file: 

http://developer.samsunq.com/android/tools-sdks/Samsunq-Andorid-USB- 

Driver-for-Windows 

•  Java  JDK  (x86  or  32-bit  version) 

o  Visit  the  following  website  to  download  the  file: 

http://www.oracle.com/technetwork/iava/iavase/downloads/idk7- 

downloads-1 880260.html 

•  Java  Cryptography  Extension  (JCE)  Unlimited  Strength  Jurisdiction  Policy  files 

o  Visit  the  following  website  to  download  the  zipped  file: 

http://www.oracle.com/technetwork/iava/iavase/downloads/ice-7- 

download-4321 24.html 
o  Navigate  to  the  following  directory: 

C:/Program  Files/Java. 

o  Open  the  most  recently  updated  jre  and/or  jdk  folder, 
o  Open  the  lib  folder  then  open  the  security  folder. 
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o  Copy  the  local_policy.jar  and  US_export_policy.jar  files  located  in  the 
unzipped  DANA  install  filers  to  the  security  folder  above. 

•  Android  SDK 

o  Visit  the  following  website  to  download  the  file: 

http://developer.android.com/sdk/index.html 
o  At  the  site  select  Download  for  other  platforms, 

o  Select  the  Windows  Installer  option  (installer_r22.0.1 -windows.exe) 
of  the  SDK  Tools  only  section  of  the  table  to  install. 

•  Android  SDK  Platform  Tools 

o  The  following  packages  will  need  to  be  installed  from  the  Android  SDK 
Manager: 

■  Android  SDK  Platform-tools, 

■  Android  SDK  Tools, 

■  Android  SDK  Build-tools, 

■  Google  USB  Driver. 

•  DANA  Data  Manager  vl .  1 . 1 1 

o  Launch  Install  DANA  Data  Manager.exe  file  from  the  unzipped  DNAN 
install  filers. 

DANA  is  programmed  to  run  in  Android,  therefore  the  mobile  computer,  Samsung 
GALAXY  Note®  vlO.1 ,  is  a  sufficient  device  for  the  purpose  of  this  SOP.  This  SOP 
requires  that  the  DANA  application  and  test  batteries  already  be  installed  on  the 
Notebook  and  all  Administrators  be  created  and  added  to  the  Notebook  using  the  DANA 
v.1.5.3  Quick  Reference  Guide  Section  C.  Administrators  may  only  be  created  and 
added  once  during  initial  setup  and  cannot  be  added  at  a  later  time  without  uninstalling 
DANA  and  erasing  the  data  from  the  Notebook.  A  USB  cable  and  Administrator  rights 
(password  protected)  are  also  required  for  transferring  data  from  the  mobile  computer  to 
the  Data  Management  PC,  opening  datasets  on  the  Data  Management  PC,  and  setting 
up  the  mobile  computer. 

Before  administering  a  screening,  it  is  highly  suggested  that  the  Notebook  settings  be 
set  to  the  following: 

•  Connections  Tab 

o  Turn  Wi-Fi  off. 
o  Turn  Bluetooth  off. 
o  Check  airplane  mode, 
o  Location  Services, 

■  Uncheck  Access  to  my  location, 
o  Turn  Nearby  Services  off. 
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Device  Tab 
o  Sound 

■  Set  Volume  to  lowest  settings, 

■  Set  Vibration  Intensity  to  highest  settings, 

■  Uncheck  and  silent  all  Notifications  and  System  sounds, 
o  Display 

■  Set  Brightness  to  highest  setting, 

■  Set  screen  timeout  to  highest  setting, 

■  Set  Touch  key  light  duration  to  Always  off, 

■  Set  Daydream  to  Activate  screen  saver  off. 

Controls 

o  Turn  Voice  control  off. 
o  S  Pen 

■  Check  T urn  off  pen  detection, 

■  Turn  pointer  off, 

■  Turn  all  Air  view  settings  off. 
o  Turn  all  Motion  control  off. 

o  Turn  all  Smart  screen  settings  off. 

General 

o  Turn  all  Backup  and  reset  settings  off 
o  Developer  options 

■  Turn  USB  debugging  on, 

■  T urn  all  other  settings  off. 


PROCEDURES 

CREA  TE  A  NEW  EXAMINER 

1 .  Turn  on  Notebook  by  pressing  and  holding  the  top  left  button. 

2.  Select  the  Apps  Tray  icon  on  the  bottom  right  corner  of  the  screen  to  view  all 
apps  downloaded  on  the  Notebook. 

3.  Launch  DANA  by  selecting  the  App  icon  (Figure  1). 

4.  At  the  Login  screen,  press  the  menu  button  to  the  left  of  the  Home  button  on  the 
Notebook  to  select  what  information  requires  editing,  Create  Examiner  or 
Manage  Device  (Figure  2). 

a.  If  Create  Examiner  is  selected,  enter  the  Examiner  Name,  New  Password, 
Confirmed  New  Password,  and  Login  Password  Hint. 

i.  Select  Save  to  save  the  changes  made  to  the  Examiner. 

ii.  Select  View  administrators  with  access  to  this  examiner  to  generate 
a  list  of  personnel  that  are  authorized  to  use  the  data. 
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ADMINISTERING  A  SCREENING 


1 .  Enter  the  examiner  username  and  password  that  was  established  during  the 
application  process  when  prompted  to  by  the  Log  in  screen  (Figure  2). 

a.  If  the  examiner  credentials  are  incorrect  the  screen  will  display  Wrong 
name  or  password  at  the  bottom  of  the  screen. 

b.  If  a  screen  timeout  has  occurred  while  using  the  app,  the  login  screen  will 
be  displayed  after  reactivating  the  Notebook.  The  login  screen  will  display 
DANA  has  restarted  due  to  loss  of  focus  at  the  bottom  of  the  screen. 

c.  The  user  must  sign  in  again  every  time  the  app  is  closed  and  reopened. 

2.  Select  an  existing  subject  or  create  a  new  subject  by  selecting  Add  a  New 
Person  (Figure  3). 

a.  Only  the  Name  field  is  required  in  order  to  create  new  subject.  The  other 
fields  (Social  Security  Number,  Unit,  and  Injured)  are  optional. 

b.  When  selecting  the  injured  box  the  screen  prompts  the  user  to  select  a 
time  and  date  of  injury  using  a  calendar  and  24  hour  clock  (Figure  4). 

c.  Once  the  subject’s  information  has  been  entered  select  Create  New 
Person  (Figure  3). 

3.  Select  the  test  battery  (DANA  Rapid  (1.1),  DANA  Brief  [NO  S](1.1),  DANA 
Standard  [NO  S](2.0))  and  select  Start  New  Screening  (Figure  5  and  6). 

a.  Hand  the  Notebook  to  the  subject  to  complete  the  screening.  The  screen 
will  provide  directions  for  the  subject. 

b.  When  the  screening  is  complete  the  subject  will  be  prompted  to  return  the 
Notebook  to  the  examiner  (Figure  7). 

4.  Select  Screening  Complete  at  the  bottom  of  the  screen  and  the  examiner  will 
automatically  be  logged  out  for  data  security. 


Figure  1 


t  t  mi  12:0? 

Hr  DANA 

WJ  X  1 12  0/ 

[f5]  DANA 

Upppi 

Maine 

Name 

Subject  1 

Defense  Automated  Neurobehawcraj  Assessment 

Version  1.5.3 

Examiner 

Password 

Soria)  Security  M  umber 

Unit 

Injured 

Social  Security  Number 

111223333 

Unit 

Bravo 

y  Injured 

Injury  Date 
[6-7-701 3 

Injury  Time 
[16  28 

Create  Examiner 

Manage  Device 

Create  Mew  Person 

Save  Person 

Figure  2 

Figure  3 

Figure  4 

A4 


Figure  5  Figure  6  Figure  7  Figure  8 


EDIT  EXAMINER  AND  SUBJECT  INFORMATION 

1 .  After  entering  the  examiner  username  and  password,  select  the  subject  whose 
information  requires  editing. 

2.  Press  the  menu  button  to  the  left  of  the  Home  button  on  the  Notebook  to  select 
what  information  requires  editing,  Edit  Examiner  or  Edit  Patient:  Subject  Name 
(Figure  8). 

a.  If  Edit  Examiner  is  selected,  the  program  will  allow  the  user  to  change  the 
Examiner  name  and  password  as  long  as  the  current  password  can  be 
provided  and  the  new  password  can  be  confirmed. 

i.  Select  Save  to  save  the  changes  made  to  the  Examiner. 

ii.  Select  View  administrators  with  access  to  this  examiner  to  generate 
a  list  of  personnel  that  are  authorized  to  use  the  examiner  data. 

b.  If  Edit  Patient:  Subject  Name  is  selected,  the  program  will  return  the  user 
to  the  subject  information  page  created  in  step  5  of  ADMINISTERING  A 
SCREENING  (Figure  4).  This  will  allow  the  user  to  make  changes  to  the 
subject’s  Name,  Social  Security  Number,  Unit,  or  Injured  information. 

i.  Select  Save  Person  to  save  the  changes  made  to  the  subject’s 
information  (Figure  4). 

ABORTED  SCREENINGS 

A  screening  is  aborted  by  pressing  the  home  or  power  button  on  the  Notebook  during  a 
screening.  A  subject  with  any  aborted  screening  will  have  a  red  warning  symbol  next  to 
their  name,  test,  and  screening  that  was  aborted. 

1 .  Select  the  aborted  subject,  test,  and  screening  (Figure  9,  1 0,  and  11). 

2.  If  no  sections  of  the  screening  have  been  completed  before  the  screening  was 
aborted,  select  Discard  Screening  or  Resume  Screening  at  the  bottom  of  the 
screen  (Figure  12). 
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a.  If  Discard  Screening  is  selected,  the  program  will  ask  Discard  this 
screening?  Select  Yes  to  delete  from  memory  and  No  to  return  back  to  the 
aborted  screening  page. 

b.  If  Resume  Screening  is  selected,  the  subject  will  restart  the  screening 
process. 

3.  If  sections  of  the  screening  have  been  completed  before  the  screening  was 
aborted,  select  Convert  to  Final  Result,  Resume  Screening,  or  Resume 
Screening  and  Repeat  Last  Completed  Section  (Figure  13). 

a.  If  Convert  to  Final  Result  is  selected,  a  report  will  be  generated  based  on 
the  section  or  sections  that  were  completed. 

b.  If  Resume  Screening  is  selected,  the  subject  will  start  screening  after  the 
last  completed  section. 

c.  If  Resume  Screening  and  Repeat  Last  Completed  Section  is  selected,  the 

subject  will  repeat  screening  of  the  section  that  was  completed  before  the 
screening  was  aborted. _ 


Figure  9  Figure  10  Figure  11 
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VIEWING  RESULTS  ON  NOTEBOOK 


1 .  Complete  screening  as  described  in  step  7  of  ADMINISTERING  A  SCREENING. 

2.  Enter  examiner  password  if  viewing  results  directly  after  the  completion  of  a 
screening. 

a.  If  viewing  results  after  closing  DANA,  log  in  as  described  in  steps  3  and  4 
of  ADMINISTERING  A  SCREENING. 

3.  Select  the  individual  screening  to  view  and  generate  the  Report  Summary  screen 
if  viewing  results  directly  after  the  completion  of  a  screening  (Figure  14). 

a.  If  viewing  results  after  closing  DANA,  select  desired  subject,  test  battery, 
and  individual  screening. 

4.  Press  the  menu  button  to  the  left  of  the  Home  button  on  the  Notebook  to  select 
sections  of  the  report. 

a.  If  selecting  a  screening  from  a  DANA  Rapid  battery  test,  the  sections  are 
Summary  and  Results  and  Cognitive  Section  Details  (Figure  15). 

i.  If  Summary  and  Results  is  selected,  the  program  will  return  the 
user  to  the  Report  Summary  screen  (Figure  14). 

ii.  If  Cognitive  Section  Details  is  selected,  the  program  will  ask  which 
section  details  the  user  would  like  to  view  Reaction  Time, 
Procedural  Reaction  Time,  or  Go  No  Go  (Figure  16). 

b.  If  selecting  a  screening  from  a  DANA  Brief  [NO  S]  battery  test,  the 
sections  are  Summary  and  Result,  Psychological  Section  Details  and 
Cognitive  Section  Details. 

i.  If  Summary  and  Results  is  selected,  the  program  will  return  the 
user  to  the  Report  Summary  screen  (Figure  14). 

ii.  If  Psychological  Section  Details  is  selected,  the  program  will  ask 
which  section  details  the  user  would  like  to  view  PHQ-8,  PCPTSD 
Survey,  or  ISI  Survey. 

iii.  If  Cognitive  Section  Details  is  selected,  the  program  will  ask  which 
section  details  the  user  would  like  to  view  Reaction  Time,  Code 
Sub  [Learning],  Procedural  Reaction  Time,  Spatial  Processing,  Go 
No  Go,  Code  Sub  [Recall],  or  Reaction  Time. 

c.  If  selecting  a  screening  from  a  DANA  Standard  [NO  S]  battery  test,  the 
sections  are  Summary  and  Result,  Psychological  Section  Details  and 
Cognitive  Section  Details. 

i.  If  Summary  and  Results  is  selected,  the  program  will  return  the 
user  to  the  Report  Summary  screen  (Figure  14). 

ii.  If  Psychological  Section  Details  is  selected,  the  program  will  ask 
which  section  details  the  user  would  like  to  view  CES  Survey,  PHQ- 
8,  PSQI  Survey,  PCLM  Survey,  or  DSI  [NO  S], 

iii.  If  Cognitive  Section  Details  is  selected,  the  program  will  ask  which 
section  details  the  user  would  like  to  view  Reaction  Time,  Code 
Sub  [Learning],  Procedural  Reaction  Time,  Spatial  Processing,  Go 
No  Go,  Code  Sub  [Recall],  Matching  To  Sample  or  Reaction  Time. 
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d. 


Note  that  if  Aborted  screenings  were  converted  into  final  results  as  in  step 
3  of  ABORTED  SCREENINGS,  only  the  sections  completed  will  have 
result  details. 
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Figure  14  Figure  15  Figure  16 


TRANSFER  DATA  TO  DATA  MANAGEMENT  PC 

1 .  Turn  on  Notebook  by  pressing  and  holding  the  top  left  button. 

2.  Plug  in  Notebook  into  the  Data  Management  PC  using  the  USB  cable  provided. 

3.  Launch  the  DANA  Data  Manager  (DDM)  on  the  Data  Management  PC  and  select 
Manage  Administrators  (Figure  17). 

4.  Select  Open  Administrator  from  the  DANA  Administrator  window  (Figure  18). 

5.  Locate  and  select  the  Administrator  Key  Store  file  (.DANAKeyStore)  created 
during  set  up  and  creation  of  Administrators  and  select  Open. 

6.  Enter  the  Administrator  password  into  the  Administrator  password  window  and 
select  OK. 

a.  If  an  Administrator  has  been  successfully  opened,  the  Administrator 
should  appear  in  the  DANA  Administrator  Manager  window  under  Current 
Administrators.  Close  the  DANA  Administrator  Manager  window. 

b.  If  an  Administrator  has  not  been  successfully  opened,  a  pop-up  window 
will  appear  displaying  Could  not  open  administrator.  Close  the  DDM 
window  and  DANA  Administrator  Manager  window  and  repeat  steps  3 
through  6. 

7.  Launch  DANA  as  described  in  steps  2  through  4  of  CREATE  A  NEW 
EXAMINER. 
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a.  If  Manage  Device  is  selected  from  the  menu  options  of  the  login  screen 
the  screen  will  display  Waiting  for  connection  from  host  computer  (Figure 
19). 

b.  If  the  menu  window  does  not  display,  press  the  back  button  to  refresh  the 
login  screen  and  repeat  step  7. 

8.  Select  Manage  Device  / Download  Data  in  the  DDM  window  on  the  Data 
Management  PC  (Figure  17). 

a.  If  the  Data  Management  PC  successfully  connects  to  the  mobile  device, 
the  screen  on  the  Notebook  will  display  Management  system  active  and 
the  Manage  Device  /  Import  Data  window  will  appear  displaying  the 
examiners  that  the  Administrator  has  access  to  (Figure  20). 

b.  If  the  Data  Management  PC  does  not  successfully  connect  to  the  mobile 
device  see  the  TROUBLESHOOTING  section  of  this  SOP  for  solutions. 

9.  Select  the  desired  Examiners  whose  data  requires  transferring  or  select  Select 
All  Examiners  in  the  Manage  Device  /  Import  Data  window  and  select  Transfer 
Data  For  Selected  Examiners  (Figure  20). 

10.  To  delete  data  on  device  after  transfer,  uncheck  the  box  that  displays  Keep  data 
on  device  after  transfer  (Figure  20). 

1 1 .  Navigate  through  the  folders  of  the  Data  Management  PC  to  select  the  desired 
location  to  save  the  transferred  data  in  the  Data  Transfer  Destination  window  and 
select  Open. 

12.  A  Message  window  will  appear  displaying  Transfer  Completed.  Select  OK. 
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VIEWING  RESULTS  ON  THE  DATA  MANAGEMENT  PC 

1 .  Launch  the  DDM  on  the  Data  Management  PC  and  select  Open  Dataset  (Figure 
17). 

2.  Navigate  through  the  folders  to  locate  the  DANABASE  file  (.danaBase)  that  was 
saved  in  step  1 1  of  TRANSFER  DATA  TO  DATA  MANAGEMENT  PC  in  the 
Choose  DANA  Base  window. 

3.  Select  the  desired  file  and  select  Open.  Files  are  named  with  the  device  number, 
date  and  time  that  the  information  was  transferred  to  the  Data  Management  PC 
as  default. 

4.  A  password  may  or  may  not  be  required  to  view  the  results. 

a.  If  a  password  is  required,  enter  the  password  for  the  Examiner  or  open 
with  an  Administrator. 

i.  To  open  with  Examiner  password,  enter  the  password  for  the 
Examiner  under  whom  the  data  was  collected  and  select  Open 
(Figure  21). 

ii.  To  open  with  an  Administrator,  select  Manage  Administrators  and 
complete  steps  4  through  6  of  TRANSFER  DATA  TO  DATA 
MANAGEMENT  PC.  After  the  Administrator  has  been  opened, 
close  the  DANA  Administrator  Manager  window  and  select  Open 
with  Administrator  in  the  password  window  (Figure  21 ). 

b.  If  a  password  is  not  required,  an  Administrator  with  access  to  the 
Examiner  results  has  already  been  opened  and  the  DANA  Viewer  window 
will  open  (Figure  22). 

5.  There  are  two  view  options  for  viewing  the  data  under  the  examiner  selected 
Tree  and  Table. 
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a.  The  default  view  option  is  Tree  with  two  folders,  Screenings  and  Patients 
displayed  in  the  left  panel  (Figure  22). 

i.  To  select  data  by  screening,  expand  the  following  folders  in  order 
by  double  clicking:  Screenings  —>  Test  Battery  — ►  Patients  —> 

Patient  Name  (Figure  22) 

ii.  To  select  data  by  patient,  expand  the  following  folders  in  order  by 
double  clicking:  Patients  — >  Patient  Name  — ►  Screenings  — ►  Test 
Battery  (Figure  22). 

b.  To  view  data  in  Table  format,  select  View  from  the  top  left  menu  and 
check  as  Table. 

i.  The  Table  format  gives  the  name  of  the  subject  and  date  of 

screening.  The  highlighted  rows  of  the  table  are  the  test  battery  and 
subtests  (Figure  23). 

6.  The  screenings  are  labeled  by  date  and  time  the  test  was  administered.  Double 
click  on  the  screening  to  view  the  report  displayed  in  the  right  panel. 

7.  There  are  two  format  options  for  the  report  Full  and  Basic. 

a.  The  default  format  option  is  Full  with  a  detailed  report  on  the  screening. 

To  expand  the  sections  and  subsections  of  Battery  Test  click  next  to  the 
little  black  triangles  (Figure  24). 

b.  To  view  results  in  Basic  format,  select  Format  the  top  left  menu  and  check 
Basic  (Figure  25). 

8.  There  are  three  options  to  view  other  DANABase  files  from  the  DANA  Viewer 
window.  To  open  an  additional  DANABase  file  select  File  from  the  top  left  menu 
and  select  New  DANABase,  Open  DANABase,  Open  Recent  DANABase,  or 
Import  Data  from  local  DANABases. 

a.  If  New  DANABase  or  Open  DANABase  are  selected,  a  Choose  DANA 
Base  window  will  appear.  Repeat  steps  2  through  4  to  open  the  desired 
DANABase  file. 

b.  If  Open  Recent  DANABase  is  selected,  a  list  or  DANABase  files  that  have 
been  recently  opened  will  appear.  Select  the  desired  DANABase  file  to  be 
opened  from  the  list. 

c.  If  Import  Data  from  local  DANABases  is  selected,  a  password  for 
DANABase  to  import  is  required.  Enter  the  password  and  select  Import 
DANABase.  Select  the  desired  DANA  file(s)  to  import  and  select  Open 
(Figure  26). 
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Figure  26 


EXPORT  TRANSFERRED  DATA 

1 .  Complete  steps  1  through  6  of  TRANSFER  DATA  TO  THE  DATA 
MANAGEMENT  PC  for  the  desired  screening  to  be  exported  (data  must  already 
be  transferred  to  the  Data  Management  PC). 

2.  Once  a  screening  has  been  selected,  select  Export  from  the  top  left  menu  and 
choose  an  export  option. 

a.  If  export  option  XML  is  selected,  one  XML  file  per  test  per  subject  will  be 
generated  including  cognitive  test  data,  psychological  test  data,  and  trial- 
by-trial  data. 

b.  If  export  option  Full  Report  is  selected,  one  HTML  file  per  test  per  subject 
will  be  generated  including  cognitive  test  data,  psychological  test  data, 
and  trial-by-trial  data. 

c.  If  export  option  PDF  Report  is  selected,  one  PDF  file  per  test  per  subject 
will  be  generated  including  cognitive  test  data,  psychological  test  data, 
and  trial-by-trial  data. 

d.  If  export  option  Test  Statistics  is  selected,  3  to  7  CSV  files  per  cognitive 
test  will  be  generated  including  cognitive  test  data. 

e.  If  export  option  Test  Responses  is  selected,  3  to  7  CSV  files  per  cognitive 
test  will  be  generated  including  cognitive  test  data  and  trial-by-trial  data. 

f.  If  export  option  Single  Row  per  Subject  is  selected,  one  CSV  file  per 
subject  will  be  generated  including  cognitive  test  data  a  psychological  test 
data  (only  scores). 

g.  If  export  option  Demographics  is  selected,  one  CSV  file  will  be  generated 
including  trial-by-trial  data  taken  from  the  demographics  survey. 

h.  If  export  option  Extended  Format  -  User  UUID  is  selected,  7  to  25  CSV 
files  (2  files  per  test  plus  1  to  3  files  with  overall  screening  information)  will 
be  generated  including  cognitive  test  data,  psychological  test  data,  and 
trial-by-trial  data. 

i.  If  export  option  Extended  Format  is  selected,  4  to  19  CSV  files  (1  to  2 
files  per  test  plus  1  to  3  files  with  overall  screening  information)  will  be 
generated  including  cognitive  test  data,  psychological  test  data,  and  trial- 
by-trial  data. 
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j.  If  export  option  Finger  Tapping  Test  Export  is  selected,  2  CSV  files  will  be 
generated  including  cognitive  test  data  (only  responses  to  finger  tapping 
test). 

k.  If  export  option  Raw  Balance  Data  is  selected,  5  CSV  files  per  stance  will 
be  generated  from  data  collected  during  the  Balance  Test. 

l.  If  export  option  Raw  Data  is  selected,  13  to  22  CSV  files  will  be  generated 
including  cognitive  test  data,  psychological  test  data,  and  trial-by-trial  data. 

3.  To  add  a  file  to  export,  highlight  the  desired  screening  in  the  table  of  the  left 
panel  and  select  Add  to  Export.  The  screening  will  be  moved  over  to  the  Export 
tab  of  the  right  panel  under  Items  in  Export  (Figure  27). 

4.  To  remove  a  screening  from  the  Items  in  Export  list,  highlight  the  desired 
screening  and  select  Remove  Item  located  in  the  Export  tab  of  the  right  panel. 

5.  Configuration  options  are  available  for  export  options  Extended  Format  -  User 
UUID  and  Extended  Format.  To  select  the  configuration  option,  check  Export 
Test  Meta  Data  (Figure  28). 

6.  Once  the  desired  screening(s)  for  export  have  been  selected  into  the  Items  in 
Export  list,  select  Export  at  the  bottom  of  the  Export  tab  in  the  DANA  Viewer 
window  (Figure  27). 

7.  Navigate  through  the  folders  in  the  Select  Export  Destination  window  to  export 
the  screenings  to  and  select  Open. 

8.  A  message  window  will  appear  stating  if  the  export  was  successful  or  not. 

a.  If  the  export  was  successful,  the  message  will  display  Export  completed 
OK;  select  OK. 

b.  If  the  export  was  unsuccessful,  the  message  will  display  Export  failed  or 
was  canceled;  select  OK. 

i.  Select  Close  at  the  bottom  of  the  Export  tab  to  close  the  tab. 

Repeat  step  2  to  select  an  export  option  that  is  more  suitable  for 
the  type  of  battery  test  and  screening  that  is  being  exported.  For 
more  details  on  the  export  options  see  Section  D-6  of  DANA  vl  .5.3 
Quick  Reference  Guide. 
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QUALITY  CONTROL: 

BACKUP  TRANSFERRED  DATA 

The  location  selected  for  the  backup  will  have  a  new  folder  with  the  device’s  serial 
number  as  the  name.  Within  this  folder  is  a  compressed  zipped  file  labeled  with  the  date 
and  time  in  which  the  backup  was  completed.  The  compressed  file  contains  Data  Base 
files,  HTML  Documents,  XML  Documents  and  HD  files. 


1 .  Complete  steps  1  through  8  of  TRANSFER  DATA  TO  THE  DATA 
MANAGEMENT  PC  for  the  desired  data  to  be  backed  up. 

2.  Select  the  examiner(s)  for  the  given  Administrator  whose  data  requires  a  backup 
or  select  Select  All  Examiners  in  the  Manage  Device  /  Import  Data  window  and 
select  Next  (Figure  20). 

3.  In  the  next  Manage  Device  /  Import  Data  window  select  Create  Device  Backup 
(Figure  29). 

4.  Navigate  through  the  folders  in  the  Backup  Destination  window  to  backup  the 
data  from  the  selected  examiners  to  and  select  Open. 

5.  A  message  window  will  appear  stating  if  the  backup  was  successful  or  not 

a.  If  the  backup  was  successful,  a  message  window  will  display  Backup 
Completed,  connection  will  close.  Select  Ok  to  return  to  the  Manage 
Device  /  Import  Data  window. 
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6.  The  connection  between  the  Notebook  and  the  DDM  will  be  lost  at  the 
completion  of  the  backup. 

TROUBLESHOOTING 

The  DDM  is  unable  to  recognize  the  Notebook  as  a  mobile  device  and/or  connection  to 
the  mobile  device  was  unsuccessful.  Connection  can  be  lost  if  the  Data  Management 
PC  is  set  to  hibernate  after  a  period  of  inactivity,  the  DDM  window  closes  Utilize  the 
following  checklist  to  ensure  a  successful  connection  between  the  Nomad  and  the 
DDM. 

□  Verify  that  the  versions  of  DANA,  the  DDM,  and  Android  are  compatible  with  one 
another.  For  the  purpose  of  this  SOP  the  following  compatible  versions  were 
used:  DANA  v.1.5.3,  DDM  vl.1.11,  and  Android  v4.3.  To  verify  the  compatibility 
of  other  versions  see  Section  E  of  DANA  vl  .5.3  Quick  Reference  Guide. 

o  The  version  of  DANA  can  be  checked  by  launching  DANA  on  the 
Notebook.  The  version  is  displayed  below  the  logo  on  the  login  screen 
(Figure  2). 

o  The  version  of  DDM  can  be  checked  by  launching  the  DDM  on  the  Data 
management  PC.  The  version  is  displayed  below  the  DANA  logo  in  the 
Dana  Data  Manager  window  (Figure  17). 

o  The  version  of  Android  can  be  checked  on  the  Notebook  by  going  to  the 
settings  app  and  selecting  About  device  at  the  bottom  of  the  settings  list 
on  the  General  tab. 

□  Verify  that  the  correct  USB  cable  provided  is  securely  connected  to  both  the  Data 
Management  PC  and  Notebook. 

o  Unplug  the  USB  cable  from  the  Data  Management  PC  and  plug  back  in  or 
use  a  different  USB  cable. 

□  Verify  the  settings  configuration  listed  in  Section  E  Equipment  and  Supplies  for 
the  Notebook. 

o  Verify  that  the  screen  timeout  is  at  the  highest  setting.  The  device  will  not 
connect  if  the  screen  times  out  before  a  connection  had  been  made. 

o  Verify  that  the  USB  debugging  is  enabled. 

□  Verify  the  installations  required  in  Section  E  Equipment  and  Supplies  for  the  Data 
Management  PC. 

o  Verify  that  the  Android  SDK  packages  Android  SDK  Tools,  Android  SDK 
Platform- tools,  Android  SDK  Build- tools,  and  Google  USB  Driver  are 
installed  and  up  to  date. 

■  If  these  packages  display  Not  installed  in  the  status  column  of  the 

Android  SDK  Manager  window,  check  the  packages  that  require 
installation  and  select  Install _ packages  (Figure  30). 

■  If  these  packages  display  Update  available  in  the  status  column  of 

the  Android  SDK  Manager  window,  check  the  packages  that 
require  installation  and  select  Update _ packages. 
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o  Verify  that  the  correct  mobile  computer  driver  is  installed  properly  on  the 
Data  Management  PC.  An  appropriate  driver  for  the  Notebook  is  the 
Samsung  Android  USB  Driver  for  Windows. 

■  Without  the  proper  USB  driver  installed,  the  Data  Management  PC 
will  be  unable  to  recognize  the  Nomad  device.  This  is  characterized 
by  the  following: 

•  The  Manage  Device  /  Import  Data  window  will  display  no 
Device  code  and  Select  a  device  for  management  or  data 
transfer  w\\h  no  detected  Android  devices  listed  (Figure  31). 

•  The  screen  on  the  Notebook  will  continue  to  display  Waiting 
for  connection  from  host  computer  (Figure  1 9). 

□  Verify  that  the  correct  Administrator  is  open  in  the  DDM.  Once  an  Examiner  has 
already  been  created  under  an  Administrator,  the  device  will  no  longer  accept 
any  additional  Administrators  unless  DANA  is  uninstalled  and  then  reinstalled 
resulting  in  the  loss  of  all  data  on  the  Notebook. 

o  Verify  that  the  Administrator  has  access  to  the  data  under  the  desired 
Examiner. 

■  Complete  steps  1  through  2  a.  of  EDIT  EXAMINER  AND  SUBJECT 
INFORMATION 

■  Select  View  administrators  with  access  to  this  examiner  to  generate 
a  list  of  personnel  that  are  authorized  to  use  the  examiner  data. 

■  Compare  the  list  of  administrators  and  signature  keys  to  the 
administrators  opened  in  the  DANA  Administrator  Manager  window 
of  the  DDM. 

■  Without  the  correct  Administrator  opened  in  the  DDM,  the  Data 
Management  PC  will  be  able  to  recognize  the  Notebook  device  but 
will  be  unable  to  connect  to  it.  This  is  characterized  by  the 
following: 

•  The  Manage  Device  /  Import  Data  window  will  display  the 
Device  code  and  Could  not  connect  to  the  device  (Figure32). 

•  The  screen  on  the  Notebook  will  display  Host  computer  may 
not  be  authorized  to  connect  to  this  device  or  Waiting  for 
connection  from  host  computer. 

1 .  To  open  the  correct  Administrator  with  authorization  to  access  the  desired 
Examiner  data,  select  Open  Administrator  in  the  Manage  Device  /  Import 
Data  window. 

2.  Complete  steps  5  and  6  of  TRANSFER  DATA  TO  DATA  MANAGEMENT 
PC  and  close  out  of  the  DANA  Administrator  Manager  window. 

3.  Select  Connect  to  device  in  the  Manage  Device  /  Import  Data  window 
(Figure  32). 

□  Verify  that  the  Notebook  is  on  Manage  Device  screen. 

o  If  the  Notebook  is  not  on  the  Manage  Device  screen,  the  Data 

Management  PC  will  be  able  to  recognize  the  device  but  will  be  unable  to 
connect  to  it.  This  is  characterized  by  the  following: 
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■  The  Manage  Device  /  Import  Data  window  will  display  the  Device 
code  and  Could  not  connect  to  the  device  (Figure32). 

1 .  To  get  to  the  Manage  device  screen  on  the  Notebook,  complete  step  7  of 
TRANSFER  DA  TA  TO  DA  TA  MAN  A  GEM  ENT  PC. 

2.  Select  Connect  to  device  in  the  Manage  Device  /  Import  Data  window 
(Figure  32). 
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Figure  32 
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